Backends - Rustic Core

A backend is rustic_core’s abstraction layer for storing repository data. Backends enable rustic to work with diverse storage systems while maintaining the same repository format.

What is a Backend?

Backends provide a simple interface to storage:

Read Operations

List files by type
Read full files
Read partial data (ranges)
Check file sizes

Write Operations

Write files
Delete files
Create repository structure

All backends provide the same interface, so switching storage doesn’t require application changes.

Backend Traits

rustic_core defines two core traits:

ReadBackend

pub trait ReadBackend: Send + Sync {
    /// Get backend location string
    fn location(&self) -> String;
    
    /// List all files of a given type with sizes
    fn list_with_size(&self, tpe: FileType) -> RusticResult<Vec<(Id, u32)>>;
    
    /// Read entire file
    fn read_full(&self, tpe: FileType, id: &Id) -> RusticResult<Bytes>;
    
    /// Read partial file (offset + length)
    fn read_partial(
        &self,
        tpe: FileType,
        id: &Id,
        cacheable: bool,
        offset: u32,
        length: u32,
    ) -> RusticResult<Bytes>;
    
    // ... warm-up methods
}

WriteBackend

pub trait WriteBackend: ReadBackend {
    /// Create repository structure
    fn create(&self) -> RusticResult<()>;
    
    /// Write file data
    fn write_bytes(
        &self,
        tpe: FileType,
        id: &Id,
        cacheable: bool,
        buf: Bytes,
    ) -> RusticResult<()>;
    
    /// Remove file
    fn remove(
        &self,
        tpe: FileType,
        id: &Id,
        cacheable: bool,
    ) -> RusticResult<()>;
}

WriteBackend extends ReadBackend, so all backends that support writes also support reads.

Available Backends

rustic_core supports four backend types:

Local
REST
Rclone
OpenDAL

Local filesystem backend - Stores repository on local or mounted filesystems.

use rustic_backend::LocalBackend;

let backend = LocalBackend::new("/path/to/repository")?;

Features:

Direct filesystem access
No network overhead
Simple and fast
Works with NFS, SMB, etc.

Best for:

Local backups
Network-attached storage (NAS)
Testing and development

REST backend - Connects to rustic/restic REST servers.

use rustic_backend::RestBackend;

let backend = RestBackend::new(
    "https://backup-server.example.com/repo"
)?;

Features:

HTTP(S) protocol
Authentication support
Compatible with restic REST server
TLS encryption in transit

Best for:

Remote backups
Self-hosted backup servers
Centralized backup infrastructure

Rclone backend - Uses rclone for 40+ cloud providers.

use rustic_backend::RcloneBackend;

let backend = RcloneBackend::new(
    "rclone:s3:bucket-name/path"
)?;

Supported services:

Amazon S3, B2, Azure, GCS
Dropbox, Google Drive, OneDrive
SFTP, FTP, WebDAV
40+ others via rclone

Features:

Leverage existing rclone configs
Wide cloud provider support
Rclone’s transfer optimizations

Best for:

Cloud storage without direct SDK
Existing rclone users
Services without native support

OpenDAL backend - Native support for multiple cloud providers.

use rustic_backend::OpenDALBackend;

let backend = OpenDALBackend::new(
    "s3://bucket-name/path"
)?;

Supported services:

Amazon S3
Azure Blob Storage
Google Cloud Storage
And more via OpenDAL

Features:

Native Rust implementation
No external dependencies
Optimized for each provider
Better error handling

Best for:

Cloud-native deployments
Embedded applications
When avoiding system dependencies

Choosing a Backend

Decision matrix for selecting the right backend:

Backend Comparison

Backend	Setup	Dependencies	Performance	Providers
Local	Simple	None	Fastest	Filesystem
REST	Medium	REST server	Fast	REST API
OpenDAL	Medium	None	Fast	10+ clouds
Rclone	Complex	rclone binary	Good	40+ services

Backend Configuration

Configure backends using BackendOptions:

use rustic_backend::{BackendOptions, SupportedBackend};

let opts = BackendOptions {
    repository: Some("/backup/repo".to_string()),
    repo_hot: None,  // Optional hot repository
    ..Default::default()
};

let backend = opts.to_backend()?;

Hot/Cold Backend Setup

Combine fast and slow storage:

let opts = BackendOptions {
    repository: Some("s3://cold-archive/repo".to_string()),
    repo_hot: Some("/fast/ssd/cache".to_string()),
    ..Default::default()  
};

let backends = opts.to_backends()?;
let repo = Repository::new(&repo_opts, &backends)?;

See the Repository hot/cold architecture for details.

Backend Operations

File Types

Backends organize files by type:

pub enum FileType {
    Config,    // Repository configuration
    Index,     // Blob indexes
    Key,       // Encryption keys
    Snapshot,  // Snapshot metadata  
    Pack,      // Data packs
}

Each type is stored in a separate directory:

repository/
├── config
├── keys/
├── snapshots/
├── index/
└── data/

Listing Files

// List all snapshots
let snapshots = backend.list(FileType::Snapshot)?;

for id in snapshots {
    println!("Snapshot: {}", id);
}

// List with sizes
let packs = backend.list_with_size(FileType::Pack)?;

for (id, size) in packs {
    println!("Pack {}: {} bytes", id, size);
}

Reading Data

// Read entire file
let data = backend.read_full(FileType::Snapshot, &snapshot_id)?;

// Read partial (for large pack files)
let chunk = backend.read_partial(
    FileType::Pack,
    &pack_id,
    true,      // cacheable
    offset,
    length,
)?;

Writing Data

use bytes::Bytes;

let data = Bytes::from("snapshot data");

backend.write_bytes(
    FileType::Snapshot,
    &snapshot_id,
    true,  // cacheable
    data,
)?;

Caching

rustic_core includes a sophisticated caching layer:

What Gets Cached?

Always Cached
Never Cached

✅ Small, frequently accessed files:

Snapshots
Index files
Tree blobs
Config file

These are marked cacheable: true.

❌ Large, infrequently accessed files:

Data pack files
Large blobs

These are marked cacheable: false to save disk space.

Cache Location

Default cache locations:

Platform	Default Cache Directory
Linux	`~/.cache/rustic/`
macOS	`~/Library/Caches/rustic/`
Windows	`%LOCALAPPDATA%\rustic\cache\`

Override with:

let opts = RepositoryOptions {
    cache_dir: Some("/custom/cache".into()),
    ..Default::default()
};

Or disable:

let opts = RepositoryOptions {
    no_cache: true,
    ..Default::default()  
};

Warm-Up Support

Some backends (especially cold storage) support warm-up to pre-fetch data:

// Check if warm-up is needed
if backend.needs_warm_up() {
    // Trigger warm-up for pack file
    backend.warm_up(FileType::Pack, &pack_id)?;
}

Warm-Up Commands

Configure custom warm-up commands:

let opts = RepositoryOptions {
    warm_up_command: Some(CommandInput::from_str(
        "glacier restore --archive-id %id"
    )?),
    warm_up_wait: Some(Duration::from_secs(3600)),  // Wait 1 hour
    ..Default::default()
};

Warm-up is useful for:

Glacier/Archive tier storage
Tape archives
Tiered cloud storage

Error Handling

Backends return typed errors:

match backend.read_full(FileType::Snapshot, &id) {
    Ok(data) => process_snapshot(data),
    Err(e) if e.kind() == ErrorKind::NotFound => {
        println!("Snapshot not found");
    }
    Err(e) if e.kind() == ErrorKind::Network => {
        println!("Network error, retrying...");
        retry_with_backoff()?;
    }
    Err(e) => return Err(e),
}

Backend Features

Enable backends via Cargo features:

[dependencies]
rustic_backend = { version = "0.1", features = ["opendal", "rest"] }

# Or enable all
rustic_backend = { version = "0.1", features = ["cli"] }

Available features:

opendal - OpenDAL backend (default)
rest - REST backend (default)
rclone - Rclone backend (default)
cli - Command-line support

Advanced: Custom Backends

Implement custom backends by implementing the traits:

use rustic_core::backend::{ReadBackend, WriteBackend};

struct MyBackend {
    // Your storage implementation
}

impl ReadBackend for MyBackend {
    fn location(&self) -> String {
        "my-custom-backend".to_string()
    }
    
    fn list_with_size(&self, tpe: FileType) -> RusticResult<Vec<(Id, u32)>> {
        // Your implementation
    }
    
    // ... implement other methods
}

impl WriteBackend for MyBackend {
    fn create(&self) -> RusticResult<()> {
        // Create repository structure
    }
    
    // ... implement write methods
}

Custom backends enable:

Integration with proprietary storage
Custom caching strategies
Specialized optimizations
Compliance requirements

Backend Best Practices

Connection Pooling

Reuse backend instances for better performance:

// Good: Reuse backend
let backend = Arc::new(backend);
for file in files {
    backend.write_bytes(tpe, &id, true, data)?;
}

// Bad: Create new backend each time
for file in files {
    let backend = LocalBackend::new(path)?;
    backend.write_bytes(tpe, &id, true, data)?;
}

Error Retry Logic

Implement exponential backoff for transient failures:

let mut retries = 0;
let max_retries = 3;

loop {
    match backend.read_full(tpe, &id) {
        Ok(data) => break Ok(data),
        Err(e) if retries < max_retries && e.is_transient() => {
            retries += 1;
            sleep(Duration::from_secs(2_u64.pow(retries)));
        }
        Err(e) => break Err(e),
    }
}

Parallel Operations

Backends are thread-safe (Send + Sync):

use rayon::prelude::*;

let backend = Arc::new(backend);

file_ids.par_iter().try_for_each(|id| {
    let data = backend.read_full(FileType::Pack, id)?;
    process_data(data)
})?;

Repository

How backends store repository structure

Encryption

Data encryption before backend storage

Deduplication

How backends store deduplicated packs

Snapshots

Snapshot metadata in backend storage

​What is a Backend?

Read Operations

Write Operations

​Backend Traits

​ReadBackend

​WriteBackend

​Available Backends

​Choosing a Backend

​Backend Comparison

​Backend Configuration

​Hot/Cold Backend Setup

​Backend Operations

​File Types

​Listing Files

​Reading Data

​Writing Data

​Caching

​What Gets Cached?

​Cache Location

​Warm-Up Support

​Warm-Up Commands

​Error Handling

​Backend Features

​Advanced: Custom Backends

​Backend Best Practices

​See Also

Repository

Encryption

Deduplication

Snapshots

What is a Backend?

Backend Traits

ReadBackend

WriteBackend

Available Backends

Choosing a Backend

Backend Comparison

Backend Configuration

Hot/Cold Backend Setup

Backend Operations

File Types

Listing Files

Reading Data

Writing Data

Caching

What Gets Cached?

Cache Location

Warm-Up Support

Warm-Up Commands

Error Handling

Backend Features

Advanced: Custom Backends

Backend Best Practices

See Also